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ABSTRACT 

The Utah State University Protocol Project has 
developed six self -instructional modules, each concerned with an 
important concept related to teacher language. Each module contains a 
student handbook made up primarily of recognition and application 
practice lessons based on printed classroom transcripts and an 8- to 
10-minute protocol film illustrating applications of the concepts in 
classroom settings. The criterion for success for the project called 
for 80 percent of the field-test participants to attain at least 80 
percent mastery on the three 'Criterion measures included as part of 
each protocol module. Two field tests were made of each protocol 
module with revision of the module based on the field- test results. 
The results on the final field-test indicated that more than 80 
percent of the learners had reached the criterion level of mastery on 
all 18 of the criterion measures employed in evaluating the six 
protocol modules. (Four tables of data are included.) (Author) 
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Introduction 

To date, the Utah State University Protocol Project has completed 
the development and evaluation of six Protocol modules, each dealing 
with one important concept related to teacher language. For each con- 
cept, specific teacher strategies and behaviors were identified to relate 
the concept to classroom application. For example, one of the concepts 
was emphasis. The module dealt with ways that the teacher could improve 
student learning through the use of emphasis. The three specific teacher 
behaviors covered were voice modulation, paraphrasing and cueing. 

Our experience with competency-based teacher education hac^ clearly 
indicated that helping the teacher master simple skills and apply simple 
concepts basic to teaching can bring about major changes in teacher per- 
j formance and classroom climate. Therefore, the USU project emphasized 

simple classroom applications of these language concepts. 



Description of the Module 

In completing one of our modules, the learner first reads a des- 
<^ crlption of the concept. This description includes specific teaching 

behaviors that the teacher can use to apply the concept. A brief re- 
view of research literature relating the concept to learner performance 
is also given. The learner then completes the first Recognition Practice 
Lesson. This lesson is a typed transcript of a class discussion that was 



rr»?/-- / ^ Presented at AERA annual meeting, New Orleans, Feb. 1973. 
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recorded in a regular classroom. The learner must identify teacher re 
marks in the transcript which apply the concept. The student's task 
is the same for the second Recognition Practice Lesson except that a 
time limit is imposed. He then views the Protocol Film which shows a 
teacher applying the concept in a classroom situation and again he is 
required tc identify keyed teacher remarks related to the concept. He 
then takes a Recognition Test which is designed to measure his recog- 
nition of teacher applications of the concept in a classroom situation 
The criterion for success on this measure is 80 percent correct. If 
the student fails to reach criterion, he repeats the practice lessons. 
After reaching criterion on concept recognition, he completes two Ap- 
plication Practice Lessons and Application Test. These are all based 
upon classroom transcripts. Certain key teacher remarks are omitted 
and the student is required to compose teacher remarks that apply the 
concept and fit the context of the transcribed lesson. 

Since it seemed likely that some users would prefer to adopt 
sonie of the USU modules but not others, it was decided to make each 
unit complete and independent so that the modules could be used singly 
or in any combination with other modules. We also felt that many 
college faculty members wouJj be more inclined to use the modules if 
they were self-instructional and could be completed for the most part - 
outside of the regular college class time. Therefore, the modules are 
individualized and can be completed by the learner entirely during 
out-of-class time if the instructor wishes to use them in this manner. 
There are, however, materials within each module which would permit 
the instructor to use in-class discussion, small work activity groups, 
simulation, role playing, or demonstrations if he wishes to do so. 
The Instructor's Guide suggests a variety of activities that can be 
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used to increase the student's skill in applying the concepts in simu- 
lated classroom situations. 

The Development Process 

The USU development procedure started with planning and building a 
prototype of each module. This prototype was then tested in one college 
class in order to collect both summative and formative evaluation data 
on the module. These data were analyzed and the results were used to 
revise the module. After revision, the total package of six modules was 
tested in two Educational Psychology classes, one at Utah State University 
and one at the University of Colorado. Formative evaluation data were 
collected during this second field test and were employed in making a 
final revision of each module. The main purpose of the second field 
test, however, was to collect summative evaluation data on each module 
in order to determine whether students completing the modules had 
reached the criterion level in terms of concept mastery. The USU pro- 
posal established a criterion level calling for 80 percent of the learn- 
ers to achieve 80 percent mastery on all criterion measures. 

Preliminary Field Test Results 

Throughout the field tests, three criteria were employed for evalu- 
ating each module. These were: (1) student's recognition of teacher 
use of the concept on the Protocol Film, (2) student's recognition of 
teacher use of the concept in typed transcripts of classroom discussions, 
and (3) student's application of the concept to typed transcripts of 
classroom discussion lessons. In order to obtain a large body of forma- 
tive evaluation data, however, student performance was recorded not 
only on the three criterion measures, but on all major steps in the 
learning cycle. In the preliminary field test, there were six points 
^ in the modules where fewer than 80 percent of the learners reached cri- 
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terion level. These are circled in Table 1. The specific errors made 
at these points were analyzed and corrections and/or additions were 
made to the material to correct the apparant deficiencies in the ma- 
terials. All of the failures to reach criterion occurred in the recog- 
nition phase of the modules. Most of the student's difficulty with 
thv^se materials could be traced to^ examples that were not clear-cut 
applications of the concept. All in all, the preliminary field test 
was regarded as successful. The field testing identified a number of 
deficiencies in the materials and, therefore, contributed substantially 
to revision. However, the preliminary materials were generally quite 
effective in that of the 42 points at which student performance was 
measured, 80 percent or more the learners reached the criterion level 
of mastery on 36. 

Final Field Test 

The percentage of learners in the USU and Colorado U final field 
tests who reached the 80 percent mastery level on each step of the 
Protocol modules is summarized in Table 2. Data for the USU sample 
were available at seven points for each module. However, data from 
the University of Colorado are limited to the three criterion measures. 
It will be noted that more than 80 percent of the learners reached mas- 
tery on all three criterion measures for each of the six modules. Ac- 
tually, an average of 87 percent of the field test subjects reached cri- 
terion on the Protocol Film recognition tests, 98 percent reached cri- 
terion on the written Recognition Tests and 95 percent reached criterion 
on the Applies* tion Tests. Therefore, learner performance was substantially 
above the criterion we had established. Only in Recognition Practice 
Lesson 1 of the Clarity module did fewer than 80 percent of the learners 
reach criterion. The performance of learners on this lesson was item- 
analyzed and the lesson was revised to eliminate ambiguous items. 
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Student Evaluation of the USU Materials . 

In addition to measuring learner performance on each Protocol module 
a questionnaire was developed to obtain feedback from students on their 
perceptions of the modules. This questionnaire was administered on two 
occasions during the field tests, once after the students had completed 
a single module and again after students had completed all of the mod- 
ules. 

We were particularly interested in obtaining the perceptions of 
field test subjects regarding the visual and sound quality of the USU 
Protocol films. Our films were made by recording discussion sessions 
involving teachers and elementary school pupils. These sessions were 
initially recorded on two-inch videotape. The videotapes were then 
edited and the edited tape was converted to 16inm black and white film 
using the electron beam recording process. Since some visual and audi- 
tory loss is brought about both by the editing and transferring to film, 
our films are not comparable to those produced on film from the beginning. 
The student responses on visual quality showed that 91 percent of the 
students considered the films to be of satisfactory visual quality while 
nine percent indicated that parts of the films were sufficiently poor 
visually to interfere with learning. Evaluation of sound quality by 
the field test subjects produced results that were substantially the 
same with 89 percent indicating that the sound quality was satisfactory 
while 11 percent reported that there were some pupil remarks that they 
could not hear. 

We were also interested in the students' perceptions of olack and 
whito vs. color film. Although earlier research has consistantly shown 
no difference between the learning outcomes with black and white vs. 
color film, much of this research was conducted before the widespread 



use of color motion pictures and color television. Our results on this 
question indicated that 95 percent of the subjects responded favorably 
to the use of black and white film. 

Ratings of the Protocol Modules 

Students in both the USU and U. of Colorado final field tests were 
asked to rate the Protocol modules with regard to seven different charac- 
teristics. You will note in Table 3 that the results of the ratings 
were generally favorable although a number of students felt that illus- 
trations should be added to make the materials more attractive. A sub- 
stantial number of students also felt there was too much repetition in 
the materials. Our experience with development of Minicourse handbooks 
at the Far West Laborat:>ry, however, indicated that even though some 
students complain about repetition, when the repetition is removed a 
substantial number fail to reach criterion. Since the cost of original 
illustrations is very high, we avoided the use of illustrations in the 
handbooks. However, since completing the handbooks, we have located a 
number of volumes of non-copyrighted illustrations and plan to employ 
some of these illustrations in the Protocol modu-*es we are currently 
developing. 

Comparisons with Other Education Courses. 

Since the number of hours of student time devoted to education 
courses is relatively small in most states, it seems likely that the 
adoption of Protocol materials would require abandoning some of the 
material currently making up the preservice teacher education curri- 
culum. Therefore, we decided to obtain the perceptions of the field 
test subjects with regard to educational content, relevance to teaching 
and interest level of the Protocol modules and other educational courses 
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they had taken. The average student in the field test samples had com- 
pleted three or four education courses prior to taking the Protocol 
materials. Results in Table 4 combine data from the USU preliminary 
field test and from the questionnaries given at the beginning and end 
of both the USU and the University of Colorado final field tests. It 
will be noted that 97 percent of the field test sample regarded the 
--■XratocoLma te r ia 1 s to._be^ equa^l_ or_^t tex._in educa t iona 1 content than 
other education courses they had completed. Field test students also 
perceived the Protocol materials as more relevant to teaching than 
other education courses with 98 percent of the responses rating the 
Protocols as equal or better. With regard to interest level, 88 per- 
cent of the field test subjects rated the Protocols to be as or more 
interesting than other education courses they had taken. 

Summary 

The Utah State University Protocol Project has developed six 
self-instructional modules, each concerned with an important concept 
related to teacher language. Each module contains a student handbook 
made up primarily of recognition and application practice lessons based 
on printed classroom transcripts and an eight to ten minute Protocol 
film illustrating applications of the concepts in classroom settings. 
The criterion for success for the project called for 80 percent of the 
field test participants to attain at least 80 percent mastery on the 
three criterion measures included as part of each Protocol module. 
Two field tests were made of each Protocol module with revision of the 
module based on the field test results. The results on the final field 
test indicated that more than 80 percent of the learners had reached the 
criterion level of mastery on all 18 of the criterion measures employed 
in evaluating the six Protocol modules. 



8 

In addition to these performance measures, a student questionnaire 
was administered near the end of the preliminary field test at Utah 
State University and near the beginning and end of the final field tests 
at both Utah State and University of Colorado. Student perceptions of 
the Protocol modules were generally favorable. Students regarded the 
audio and visual quality of the Protocol films to be satisfactory for 
the most part. They rated the various elements in the Protocol packages 
favorably and regarded the Protocol materials to be superior to conven- 
tional education courses they had taken in terms of quality of the 
educational content, relevance, and interest level. 

It may be concluded that the USU Protocol modules are effective in 
bringing about an understanding of certain basic concepts related to 
teacher language. These materials are also perceived favorable by 
students. Since the modules are designed for independent learning and 
can be completed by students entirely during out-of-class time, it appear 
that these modules can make an important contribution to preservice 

edu;ation without interfering with the ongoing teacher education cur- 

2 

riculum. 
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The six modules on teacher language are now being used in a number 
of teacher education programs. They are being marketed on a non-profit 
basis by Utah State University. The entire package of six modules in- 
cluding Protocol Films, Student Guides, Criterion Tests and Instructor 
Handbook is available for $120.00. For further information, write to 
Professor Walter R. Borg, Dept. of Psychology, Utah State University, 
Logan, Utah, 84321. 



TABLE 1 



Preliminary Field Test 

Percent of Learners Reaching the 80% Mastery 
Level on Each Step of the USU Protocol Modules 



Module 


Rec. 
Prac. 


Rec. a 
Prac. 


Rec.b 
Prac. 


Pro. 
Film 


Rec. 

Test 


Appl. 
Prac. 
1 


Appl.c 
Prac. 
2 


Appl.d 
Prac. 
3 


Appl. 
Test 




1 








Extension 


100 


100 




82 


© 


88 






82 


Encouragement 


96 








92 


80 






1 

100 


Clarity 




93 






98 


96 


97 




92 


Emphasis 


82 


100 




81 


100 


93 


100 




80 


Feedback 


89 


86 




82 


98 


100 


100 




100 


Organization 


98 


100 


94 


© 


100 


100 


97 


100 


100 



Q 

Recognition Practice Usson 2: This step was optional for Extension and 
E icouragetnent; for all following packages it was required. 

^ Recognition Practice Lesson 3: Included in Organization package only. 

^ Application Practice Lesson 2: This step was optional and was completed by 
fewer students. Students were instructed to complete this step if they 
did not reach criterion on Application Practice Lesson 1. 

^ Application Practice Lesson 3: Included in Organization package only. 



TABLE 2 



Final Field Tests 

Percent of Learners Reaching the 80% Correct 
Level on Each Step of the Protocol Modules 



Rec. Rec. RecTa Pro. Rec. A^. Appl.b Appl.c 

Module and Prac. Prac. Prac. Film Test Prac. Prac. .Prac tIL- 

Sample 12 3 12 3 



Extension 

Utah State 80 94 89 85 98 97 93 

U. of Colo. 81 95 100 

Encouragement 

Utah State 100 96 89 100 96 100 91 

U. of Colo. 81 100 , 100 

Clarity 

Utah State 78 98 83 100 100 100 100 

U. of Colo. 84 100 84 

Emphasis 

Utah State 100 100 92 100 100 100 100 

U. of Colo. 87 97 97 

Feedback 

Utah State 96 97 81 100 100 100 80 

U. of Colo. 92 98 100 

Organization 

Utah State 96 98 95 100 100 98 97 100 100 

°f Colo. 87 100 100 



a Recognition Practice Lesson 3: Included in Organization package only. 

^ Application Practice Lesson 2: This step was optional and was completed by 
fewer students. Students were instructed to complete this step if they 
did not reach criterion on Application Practice Lesson 1. 

c Application Practice Lesson 3: Included in Organization package only. 



TABLE 3 



Rating of the Protocol Modules 
Final Field Test, Final Questionncire 



Item 


Strongly 
Agree 


Agree 


No 

Opinion 


Disagree 


Strongly 
Disagree 


The objectives of the 
protocol modules were 
clear: 

Utah State U. 

ft ^ ^ 

U. of Colo. 


18* 

1 o 


31 
lb 


2 


1 




The materials were 
attractive and in- 
teresting. 

Utah State U. 

u. or Colo. 


8 

9 


34 

11 


10 
11 


3 
8 


3 


The content was well 
organized. 

Utah State U. 

U. of Colo. 


11 

16 


32 
19 


4 


5 




Important ideas were 
easily lecognized. 

Utah State U. 

U. of Colo. 


12 
16 


32 
17 


5 
2 


5 




Repetition of impor- 
tant content was ad- 
equate. 

Utah State U. 

U. of Colo. 


17 
12 


25 
16 


3 
2 


8 
5 


2 


The materials in the 
student handbook 
were easy to use. 

Utah State U. 

U. of Colo. 


18 
17 


31 
19 


1 


1 
1 


1 


Work sheets and vis- 
ual materials were 
well-intergrated. 

Utah State U. 

U. of Colo. 


14 
12 


31 
20 


3 
2 


2 


1 



* Number of students making this 
questions. 



response. 



AH 



students did not respond to all 



tabu: 4 



Learner Perceptions of the Educational Content, 
Relevance to Teaching, and Interest Level of 
rotocol Modules as Compared with other Education Courses* 



Protocols were 
percieved as: 


Educational 
Content 


Relevance 


Interest 
Level 




N 


% 


N 


% 


N 


% 


Much better 


34 


17 


60 


31 


29 


15 


Better 


103 


50 


89 


46 


74 


39 


About equal 


62 


30 


40 


21 


65 


34 


Not as good 


5 


2 


2 


1 


19 


10 


Much poorer 


2 


1 


2 

1 


1 


4 


2 



* Based upon data from preliminary and final field tests. 



